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Purpose of a Pro- 
gram Manual 


Supplement I to 
Recommendations for Reporting 
the Effectiveness of Programed 
Instruction Materials' 


Recommendations Concerning 


Program Manuals 


(Revised October 1965) 


JOINT COMMITTEE ON 
PROGRAMMED INSTRUCTION 
AND TEACHING MACHINES? 


It is assumed that most publishers will prepare program manuals to 
accompany published programs. Such a manual would be used by 
teachers, curriculum supervisors, or others who wish information 
about the nature of the program. This supplement is intended to help 
program producers prepare an effective manual to accompany a specific 
instructional program. 

There are many agencies, such as large school systems, that may also 
wish to prepare a program manual as either a substitute for or ampli- 
fication of the one furnished by the program producer. This supple- 
ment, therefore, is addressed to all persons who may have occasion to 
prepare a program manual for use with an instructional program. 

In some respects, the function of a program manual is similar to that 
of manuals supplied with psychological or educational tests. It should 
provide information which will help a prospective user make appro- 


* This report is a supplement to a more extensive document (published in 
the Spring 1966 issue of AV Communication Review), ‘‘Recommendations 
for Reporting the Effectiveness of Programed Instruction Materials” (Re- 
vised October 1965). 

*A committee representing the American Educational Research Associa- 
tion, the American Psychological Association, and the Department of Audio- 

(Footnote continued on p. 244) 
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Suggestions Con- 
cerning Contents 
of Program 
Manuals 


General Features of 
a Program Manual 
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priate and effective choices of programs. In addition, the manual can 
provide guidance as to the most effective application of the program. 
The manual should include the following types of information: 


(1) Content of instructional program. 

(2) Description of intended student population and tryout popula- 
tion for which test data are reported. 

(3) Rationale for tests used to assess instructional program. 

(4) Evidence of effectiveness. 

(5) Practicality (cost, etc.). 

(6) Procedures for introduction and use. 

The program manual can be prepared so that new data on the pro- 

gram’s effectiveness can be readily included in subsequent revisions 

of the manual. Each edition of a manual should be dated. Manuals 

which are prepared for preliminary, limited editions of programs, prior 

to validation by definitive evaluative studies, should be clearly identi- 

fied to the purchaser as such. 


1. Information Concerning Content of Instructional Program 

To help the user decide whether the program content is appropriate for 
his purpose, the program manual should state in detail the minimum 
objectives of the program, preferably by specifying student behaviors 
or competencies which its use is intended to achieve. The objectives 
should be exemplified by test items which are regarded as suitable for 
measuring the intended outcomes of the program. These test items 
should be drawn from a test included or described in the manual. 

It is helpful and appropriate to state objectives so that they may be 
compared with the newer as well as more conventional curricula. The 
manual might well cite comments from reviews in professional journals 
which indicate how a certain content is related to specific curriculum 
objectives. An outline of the specific content covered is often desirable. 


2. Description of Intended Student Population 
The kind of student for whom the program is designed should be spec- 
ified. Information will probably be included regarding grade level, 
cultural background, age, and prerequisite skills. Minimum compe- 
tencies necessary for success in the program might be presented by 
prerequisite scores on a number of tests. These tests might include 
scholastic aptitude, reading, and specific measures of competence in 
the subject-matter field for which the program is designed. 

The manual should make clear the nature of the population used in 
evaluations of the program. Any cultural or sociological differences 


visual Instruction (National Education Association). Committee members 


are: Harry F. Silberman, Evan R. Keislar, Robert Glaser, and Arthur A. Lums- 
daine, Chairman (AERA); Richard S. Crutchfield, James G. Holland, and 
Lawrence M. Stolurow (APA); and Jack V. Edling, Edward B. Fry, Wesley C. 
Meierhenry, and Paul R. Wendt (DAVI). The work of this committee has 
been supported by the Educational Media Branch, Office of Education, U.S. 
Department of Health, Education, and Welfare, under Title VII of the Na- 
tional Defense Education Act. 
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between the intended population and the tryout population should be 
specified. 


3. Rationale for Tests Used To Assess Instructional Program 

The manual should give a brief description of the rationale for the 
selection of the criterion test for the program. It should make explicit 
the relation of the statement of objectives to the test used in measuring 
the program’s effectiveness. Such clarification will help in the prepara- 
tion of additional tests, if desired, and will aid the program user to 
interpret existing data. 

The manual should also indicate which of the desirable outcomes in 
the program’s content area are not being developed by the program 
and thus provide a better understanding of what are reasonable expec- 
tations for the program. Such an explanation would demonstrate how 
various classes of test items were related to the kind of behavioral 
changes that reflect the objectives of the program. It is necessary not 
only to supply test questions to clarify the program’s objectives but 
also to indicate what kinds of answers are acceptable or nonacceptable. 

Where test items used to illustrate an objective are different from 
those actually used in the test adopted to measure program effective- 
ness (as reported in the manual), such differences should be explicitly 
stated. Whenever test items are used for illustrative purposes, the 
reader should be able to find the tests from which these items were 
drawn. Sometimes these tests will be reproduced in full in the manual, 
but if not, references to the appropriate documents should be given so 
they may be readily located. 


4. Information About the Program's Effectiveness 

Evidence for the effectiveness of a program should be based on care- 
fully conducted studies which show what the use of this particular 
program has accomplished under specified conditions. 

Since the program manual will be designed for the use of teachers 
and school administrators, it should report the effectiveness of the pro- 
gram in straightforward terms. The reader should not be required to 
examine and interpret detailed technical data. 

The manual should differentiate clearly and explicitly between 
(1) the opinions of experts or others about the effectiveness of the pro- 
gram and (2) documented evidence on the outcomes actually obtained 
in practice. The manual should cite the sources of the available evidence 
to document any claims made about the effectiveness of the program. 
These sources should include not only those from the producer’s own 
studies but evidence from other carefully conducted studies as well. 
Many such independent studies, for example, may be carried out by 
school systems, research projects, or other agencies which have con- 
ducted program-assessment studies. 

All claims or statements made in the manual about the effectiveness 
of a program (not directly supported in the manual) should be docu- 
mented by specific reference to evidence contained in one or more de- 
tailed technical reports prepared in keeping with accepted standards of 
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scientific writing. (Specific recommendations for the preparation of 
such documents are presented in Supplement II to this report.) 

While brief summary statements concerning program effectiveness 
are appropriate in the program manual (as well as in advertising bro- 
chures or on the cover or label of a program), such summary statements 
should always make reference to the technical report on which they 
were based. The manual should make explicit the correspondence be- 
tween interpretive statements and the underlying data. The user should 
be told how and where he can obtain a copy of the technical report even 
though the manual is a derivative of such a report. 

The manual should describe in a straightforward manner the studies 
undertaken to evaluate the program. The reader should be informed 
with respect to how the students were selected, the way in which the 
program was administered, the nature of the evaluative measures, and 
the results obtained. The description of these empirical studies should 
be nontechnical but precise. The data may be presented in simple form, 
such as through the use of graphs. 

Any data on the program’s effects on students’ attitudes should be 
accompanied by a clear statement that student interest does not neces- 
sarily indicate program effectiveness. Where attitudes are measured 
solely through verbal responses, there should be no implication that 
other behavioral indices of motivation are also affected. For example, 
students may say that they liked the program enormously, but none 
of them may volunteer to receive further instruction. 


5. Information Concerning Practicality of the Program Use 

The user needs to be able to find out from the manual whether or not 
the program would be feasible for use by him in his local situation. The 
manual, therefore, can tell the reader whether the programs are re- 
usable, and, if so, how many times they might reasonably be reused. 
Where supplemental material or equipment is needed, these should be 
described along with statements of initial and maintenance costs. In- 
formation should be available to the reader about supervision require- 
ments for the students taking the program. It is desirable to know the 
median and range of training time required to present the program 
effectively. The reports of empirical studies should be complete enough 
to provide information of this kind. In short, the user should be able 
to determine within reasonable limits the cost of instructing students 
by means of the program. 


6. Procedures for Introduction and Use of the Program 

The user should be given a clear picture of the instructional conditions 
necessary for the success of the program. The manual should present 
a clear and detailed description of the recommended procedures for 
introducing students to the program and for administering the instruc- 
tional and evaluative activities. 
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